Skip to content

feat(web): add web generator #285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat(web): add web generator #285

wants to merge 4 commits into from

Conversation

avivkeller
Copy link
Member

@avivkeller avivkeller commented May 28, 2025

Fixes #7.

This PR adds the web generator.

Tasks / Issues

P1 – Must Complete Before Merge

  • Add more items (anyone can do this as they review1)

P2 – Must complete before migration

  • Add more items (anyone can do this as they review1)

P3 – Can Be Done in a Follow-up

  • Remove mustache dependency
  • Adding a tooltip/expand to the DataTag
  • Use new Orama UI
  • Add more items (anyone can do this as they review1)

Get a preview

node bin/cli.mjs generate -t orama-db -i "/node/doc/api/*.md" -o "./out" # If you want search functionality
node bin/cli.mjs generate -t web -i "/node/doc/api/*.md" -o "./out" --index "/node/doc/api/index.md" # Specifying `--index` is optional, but recommended
npx serve out # You can serve the output, or just open one of the files in your browser. Serving is required for using search.

Footnotes

  1. Add things as they appear, or leave review comments. 2 3

@codecov-commenter
Copy link

codecov-commenter commented May 28, 2025

Codecov Report

Attention: Patch coverage is 55.53914% with 602 lines in your changes missing coverage. Please review.

Project coverage is 70.59%. Comparing base (41ce248) to head (8ca0b57).

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...nerators/jsx-ast/utils/createSignatureElements.mjs 29.01% 159 Missing ⚠️
src/generators/web/index.mjs 39.55% 81 Missing ⚠️
src/generators/jsx-ast/utils/buildContent.mjs 51.72% 70 Missing ⚠️
src/generators/web/build/generate.mjs 22.36% 59 Missing ⚠️
src/generators/web/build/css.mjs 24.19% 47 Missing ⚠️
src/generators/web/build/bundle.mjs 23.21% 43 Missing ⚠️
src/generators/jsx-ast/utils/buildBarProps.mjs 36.92% 40 Missing and 1 partial ⚠️
src/generators/jsx-ast/index.mjs 26.00% 37 Missing ⚠️
src/generators/orama-db/index.mjs 54.34% 21 Missing ⚠️
src/utils/queries/index.mjs 74.00% 10 Missing and 3 partials ⚠️
... and 10 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #285      +/-   ##
==========================================
- Coverage   72.14%   70.59%   -1.56%     
==========================================
  Files         117      128      +11     
  Lines        9992    11011    +1019     
  Branches      597      644      +47     
==========================================
+ Hits         7209     7773     +564     
- Misses       2780     3231     +451     
- Partials        3        7       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@avivkeller avivkeller marked this pull request as ready for review May 28, 2025 22:13
@avivkeller avivkeller requested a review from a team as a code owner May 28, 2025 22:13
@avivkeller avivkeller marked this pull request as draft May 29, 2025 16:36
@avivkeller
Copy link
Member Author

avivkeller commented May 30, 2025

🎉 The code now dehydrates to the client so it can render without JavaScript!

@AugustinMauroy
Copy link
Member

🎉 The code now dehydrates to the client so it run without JavaScript!

Wow 😵‍💫 and what about codetab

@avivkeller
Copy link
Member Author

avivkeller commented May 30, 2025

It rehydrates and runs with JS, but if you don't have JS, you can still view the docs. I used React's SSRing

@avivkeller
Copy link
Member Author

@AugustinMauroy and I got search to finally work 🎉
image

@avivkeller avivkeller force-pushed the feat/web/gen branch 2 times, most recently from 024bbea to dbfe55d Compare June 3, 2025 21:40
@avivkeller
Copy link
Member Author

@nodejs/nodejs-website @nodejs/web-infra ChangeHistory and SideBar aren't implemented yet, so this is still a draft, but it's ready for you to take a look, so feel free to review :-)

@avivkeller avivkeller linked an issue Jun 6, 2025 that may be closed by this pull request
@ovflowd
Copy link
Member

ovflowd commented Jul 15, 2025

My proposed solution was to abandon it, and manually set the stabilities in node core's index.md file.

You mean manually writing down the stability of each one of the modules? Noted... But how the current tooling does that if Makefile also uses only individual files? Or does it also fail currently for individual files? I don't recall tbh.

@avivkeller
Copy link
Member Author

But how the current tooling does that if Makefile also uses only individual files? Or does it also fail currently for individual files? I don't recall tbh.

See https://github.com/nodejs/node/blob/4102dcc2269d12cb576468370419b059c31e72b0/tools/doc/stability.mjs, which uses the generated all.json file to generate index.html.

@avivkeller
Copy link
Member Author

Can we make these headers sticky? What I mean is while scrolling on the utmost parent level (i.e. class, or method) while scrolling the piece that contains the title, pill, history button and the type definition should be sticky... I think that'd make it easier for the user navigating on our huge docs what section they are even in?

I'd love to... but it's actually quite complex. If we use position: sticky, each header would need to define a background. If we use JavaScript, we'd, well need to add JavaScript. If you're fine adding more JS, then let's go for it.

@AugustinMauroy
Copy link
Member

Capture d’écran 2025-07-15 à 21 43 25 Became Capture d’écran 2025-07-15 à 21 43 47

Which is take more place. DO we have ability to detect this case and don't duplicate it or merge it

@avivkeller
Copy link
Member Author

avivkeller commented Jul 15, 2025

Which is take more place. DO we have ability to detect this case and don't duplicate it or merge it

I originally merged them, but we have so many different edge cases on node core, I'd rather follow-up with an issue adding:

  • Lint
  • Formatting Core

Before implementing such a feature. It's a lot of work to resolve the edge cases, and I don't want that to break this, and block it from merging.

@ovflowd
Copy link
Member

ovflowd commented Jul 15, 2025

Which is take more place. DO we have ability to detect this case and don't duplicate it or merge it

I originally merged them, but we have so many different edge cases on node core, I'd rather follow-up with an issue adding:

  • Lint
  • Formatting Core

Before implementing such a feature. It's a lot of work to resolve the edge cases, and I don't want that to break this, and block it from merging.

What if we only handle the non-edge cases? Would that make sense?

Also

image

nit: Can Return not be in a <code> since it is not a property, but the return value?

@avivkeller
Copy link
Member Author

What if we only handle the non-edge cases? Would that make sense?

No, I don't think that would. The edge cases are headers that have slightly different descriptions/history/whatnot, and would require additional logic to merge / not to merge. You can't, in my eyes, handle the non-edge cases without accounting for the edge-cases, so it would be easier (IMO) to handle it all in a follow-up.

@avivkeller
Copy link
Member Author

FYI I made a new milestone, and will open / am opening follow-ups

@ovflowd
Copy link
Member

ovflowd commented Jul 16, 2025

Oh hi @TheAlexLichter just wanted to randomly say thank you for chiming in here 🫶

@TheAlexLichter
Copy link

@ovflowd Sure thing! 🙌 Glad I could help ☺️

@avivkeller
Copy link
Member Author

@nodejs/web-infra Is there anything else I missed that I still need to implement. Just waiting on a CODEOWNER review from y'all. We are almost there 🚀!

Copy link
Member

@ovflowd ovflowd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feature-wise :shipit: 🚀 -- code wise, 👎

We need to add much more jsdocs, inline comments, extract regexes and things into constants and better simplify/cleanup the code.

Otherwise we will have a hug knowledge gap day one of what each logic means what it does and etc. I also appreciate exmaples on jsdocs. You can use Copilot or some Agentic stuff to help on that if it is massive 😅

[SAMPLE],
SAMPLE,
{},
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be an inline object { run: async... }

*
* @param {*} value - The value to convert to an ESTree node
*/
const toESTree = value => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure there's an utility on unist that already does that, no?

value: entry.heading.data.name,
}));
.filter(
entry => entry.heading?.data?.text && entry.heading?.data?.depth < 4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make 4 a constant and explain why 4 also put ?.text.length > 0 to make it explicit

const parsed = getVersionFromSemVer(version);

let label = `v${parsed}`;
if (isLts) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: separate assignemtns from conditions (empty lines)

}

return {
value: getVersionURL(parsed, entry.api),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: assignt o constant after line 75, then short object notation return { value, label }


let headingContent = text
? getFullName(content.data, false) ||
slice(content, (findText(content, ':')[0] ?? -1) + 1).node.children
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one line is quite convoluted, let's break this down and move to a dedicated function and document/explain what it does

* @returns {number} - 2 if node is a type connector, 1 if node is a type reference, 0 otherwise
*/
const checkPossibleType = node =>
node.type === 'text' && node.value === ' | '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be more documented / broken down

export const parseListIntoProperties = node => {
const properties = [];

for (const item of node.children) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also appreciate if this could be more documented, broken down, simplified. Huge function.

cells.push(createElement('td', prop.types.length > 0 ? prop.types : '-'));
cells.push(createElement('td', prop.desc.length > 0 ? prop.desc : '-'));

return prop.sublist
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dedicated function? or if else with push idk

}

const returnStr = returnType ? `: ${returnType.type}` : '';
const paramsStr = params
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be cleaned up

@ovflowd
Copy link
Member

ovflowd commented Jul 16, 2025

(FYI I've stopped my review-mid-term, as I've seen the same pattern of lack of docs, code that could be cleaned-up split-up etc... and then I just realised better just adding my review and letting @avivkeller work on that 🙈)

@avivkeller
Copy link
Member Author

Thanks for the review! I'll get right on it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Vercel Deployment once we have minimal CLI + Web generators ready Add react-web generator Write React Components